What if this NIP also included:

• AUTH (NIP-42) requirement for relays to return chat messages (even if relays don't implement that at first, clients should be ready for when they do).
• Explicit relay signaling: users must signal via NIP-65 that they want to receive DMs only on some specific relays, and whenever one wants to message these users they should only always send DMs to these relays and only those.

I think these two small measures will reduce metadata leak to basically zero in normal times and also reduce bandwidth significantly.

@fiatjaf sounds good. How does the new thing look like in general to you? And WDYT about #468?

minor comments, looks great

AUTH (NIP-42) requirement for relays to return chat messages (even if relays don't implement that at first, clients should be ready for when they do).

if we make AUTH (NIP-42) mandatory then we can drop NIP-59 #468 completely because there is no use for it.

making AUTH mandatory is a nonstarter, imo. also the disclaimers in this nip are misleading. if metadata is not a concern (and there's no reason we should blanket assume that it is), there is no need to use AUTH. i can think of many good reasons why you definitely want metadata to be present in this shared-custody data

also, there are other solutions ... it would be easy to gift-wrap (nip59?) or to have a system send spurious messages regularly to preventing harvesting of this info.

the enryption provided in this leaks the sender and the recipient public keys. that's the only disclaimer we need. it is correctly done now and scaring people away from using it isn't the right move.

We got to pick one option or the other to stop metadata leaks with this.

Option 1: Make NIP-42 mandatory and explicit relay signaling in the profile.
or
Option 2: Encrypt metadata as per NIP-59 but free to post on any relay.

Option 2 wastes compute and bandwidth resources but less restrictive.

But we have to do either one and close this topic for a while.

one more. I am new to this NIP process. Shouldn't @earonesty name be on the NIP-44 since he started this discussion and did initial work?

@starbackr-dev

he started this discussion

The initial discussion was started by mikedilger in #72, adiabat in #107 (w me proposing chacha on feb 11), AdamISZ in #303. I also don't know how nip authoring process works. Does anyone know?

i don't care, in the slightest, if my name is on it. i understand the tradeoffs and i think good decisions are being made, and it basically aligns with what i need. i think metadata "leakage" (if it can be called that), is outside the purview of this nip, and it should just be made clear that sender public key info is visible and that additional tags should not be used on these messages.

1. very easy to hide the sender by rolling new pubkeys (see nip 59 which i love)
2. very easy to authenticate with relays using a mechanism that isn't your public key (probably a bad idea anyway to use that)

metadata is not "leaked" ... this is a conversation that the sender can decrypt later, because nostr is an censorship resistant data store, and users want to be able to move between clients and see their outbounds too.

(technically recipient information doesn't matter and cannot matter)

I like this NIP. I think it makes the right decisions. We can't do forward-secrecy or metadata secrecy in nostr without a lot of other junk (if at all) and we need a better but still simple DM system.

Explicit relay signaling: users must signal via NIP-65 that they want to receive DMs only on some specific relays, and whenever one wants to message these users they should only always send DMs to these relays and only those.

I like this. I could specify my personal relay that doesn't allow anybody to read back what was written.

The initial discussion was started by mikedilger in #72, ...

Don't put me as an author. I didn't think you were going to, but just for clarity.

NIP-59 handle metadata secrecy elegantly and very well. It does exactly what nostr should do. We just need to update NIP-59 to specify that it uses NIP-44 encryption instead of NIP-04 and now we've got the building blocks of a a lot of great protocols!

@paulmillr Do you have an working example of the padding oracle attack?

@BlowaterNostr no

Question, how does this perform against on-the-fly modifications of the public key, signature and content?

If Mallory would change Alice's message, right before it reaches the relay, and changes the public key so it matches hers, changes the ciphertext to something else and recomputes the signature, is there any way to validate that the message actually came from Alice?

Why not pass that data into Poly1305 as AAD, so it's protected from being modified?
Or am I wrong?

@iloveyumyumshrimpnoodles how would poly1305 protect anything in this case? mallory's new ciphertext would include new poly authentication tag as well. It would validate properly.

at this point Mallory has changed the entire message to something she felt like sending

it is just a message from Mallory now

she might as well just posted it from the outset

If she's changing the content and the public key and the event hash and everything

Ah yeah, true.
My bad, haven't got much sleep

simple feature. instead of signing with his public key, a client signs with (shared-secret*privatekey) instead. then the recipient of the dm can verify the message using (pubkey*sharedsecret), but nobody knows who sent it, and also the recipient can't prove who sent it without releasing his private key. clients can calculate an authors: filter easily as well, this gets a lot of privacy with little effort, and it can be marked OPTIONAL, because the only downside is a lack of validation (clients will still show the messages)

Stealth addresses are hard and require complicated analysis to ensure discrete logarithm problem is not stupidly solvable. i've been around an audit of one such system. Better to not have it in the initial release.

Stealth addresses are hard and require complicated analysis to ensure discrete logarithm problem is not stupidly solvable. i've been around an audit of one such system. Better to not have it in the initial release.

i would hope that signing with a tweaked private key even if everyone knows how you tweaked it, cant possibly be broken or basically all derivations are broken

Sorry, @paulmillr, I forgot to answer #574 (comment).

I think "gift wrap" and all the other similar proposals that use random keys or anonymous keys are bad because for the incentives to work on Nostr relays ultimately will have to know something about who is writing to their database, so these solutions won't scale or may require additional complexities (despite already being complex and resource-intensive for clients).

How do you feel about including #574 (comment) in the NIP?

1. I've added NIP-42 and NIP-65. cc @fiatjaf. However, one case is unclear to me. What if I want to communicate with Alice, via specific, "more secure" relay that we both agreed upon, which is not in the list of our preferred relays? Also, how does this play with wrapper "hidden" messages that have been mentioned above?

2. With regards to content encoding, our options are:

   • hex, with splits at specific indexes ((0, 2), (2, 34)). Pros: consistency with sig and pubkey fields 1c35e24e540566e8464dfe2d5c43453d936207fdfe
   • base64+csv. It is 1.5x shorter than hex and can save some traffic. Disadvantages would include the need to include base64 implementation (currently no other NIP requires it, besides NIP4, which we're replacing). 1,hkMpXxHhTPZA3gFYrsJPWEfGO6YBc0Lw,5EZtlb1tF4uf

3. With regards to algorithm choice:

   • iOS has ChaCha20-Poly1305 https://developer.apple.com/documentation/cryptokit/chachapoly
   • Android has ChaCha20, ChaCha20-Poly1305 https://developer.android.com/reference/kotlin/javax/crypto/Cipher
   • HPKE rfc9180 has ChaCha20-Poly1305 https://datatracker.ietf.org/doc/rfc9180/
   • MLS has ChaCha20-Poly1305 https://datatracker.ietf.org/doc/draft-ietf-mls-protocol/
   • Sodium has box/sealedbox/secretbox xsalsa20-poly1305, secretstream xchacha20-poly1305
   • tweetnacl has xsalsa20-poly1305

Before merge, I also want to have a second opinion from someone experienced, who can do security evaluation / audit. If anyone knows such people, it would massively help.

Is metadata leakage and cryptographic security 2 several problems? Can we solve them independently?

Of course, some good designs might solve them at once, but it's not a good mentality to have to solve them at once. If we can solve one without making the other worse, client developers will catch up faster.

Recently I don't even have time to carefully evaluate this NIP despite Blowater is mainly focusing on DM.

Thinking again about this, I think people won't switch easily, so it may be better to try to implement SimpleX instead? Even though it is too complicated, someone can take an OpenSats grant and write a SimpleX JS library and libraries for other languages, lobby the SimpleX people to enable websocket support on their servers or write a separate implementation, and allow easily hosting paid SimpleX relays or community whitelisted SimpleX relays, then we have a solid feature to build upon (assuming SimpleX is actually good).

I strongly suggest we finalize NIP-44, explicitly state our threat model, do some sort of audit of the result scheme, and release it to public. This is because Simplex is extremely complicated. I will open a separate discussion with regards to Simplex estimates and amount of work, it's likely 100x more than finalizing NIP-44.

Regarding "people won't switch easily" - on a contrary, I don't think so. We just need to sell it. If it's too complex, then yes, they won't switch. If it's as simple as it gets, then switching should take no time. It's important to understand that, in the worst case, all the primitives (xchacha, whatever) are still available in extremely popular libsodium, which works on all platforms. So, it's good. For JS i've released noble-ciphers, so it's good too.

To reiterate deficiencies of NIP4: 1) cbc padding oracle attacks 2) multi-user security of AES is shit 3) cross-platform key reuse, it's easy to accidentally leak private key 4) and aes key not being IND. Maybe 4 is minor. 1 and 2 are medium. 3 is really bad, since nostr uses one key for everything.

@BlowaterNostr metadata leakage of some sort always happens. It happens in signal, it happens in tor, it happens everywhere. The best we can do is minimize it. I think fiatjaf's solution would radically decrease leakage, so it's cool.

SimpleX discussion => #658

we're using the sha256(shared secret) to sign/encrypt messages. but are otherwise using nip-44 for encryption. that prevents metadata leakage better than most of the above. and its plausibly deniable. and it's understandable/easy to use. i really would like something in nip-44 that mentions that this is an easy option to add "metadata free dm's" to any nostr client

@earonesty

and its plausibly deniable

how so?

Folks, I can't find a reliable xchacha20 implementation for Android. Do you have any tips? It's going to be hard to support this PR if the cryptographic function is not easily available in all languages. :(

@vitorpamplona libsodium is available on all platforms.

To clarify, in libsodium the function is called crypto_stream_xchacha20. There are bindings to basically all languages, that includes java, js, swift, and everything else.

I got it to work with one of the bindings: #468 (comment)

3/4 of the libs I tested were crashing for no reason. Things are still very raw.

we're using a variant of this in arcade, running on android. it doesn't use chacha, but it does the "right thing" for aes. (new salt, new key per message): https://github.com/ArcadeLabsInc/arclib/blob/9857e1ae1558b9e265c1461a53d281ecbe4fae77/src/ident.ts#L107

works fine in react/ios/andriod and is def better than nip04. also we support blinded-dms as well (no metadata leaks) on android

i had various problems using GCM and chacha, so this is where i left it

your solution is still:

• much less secure against multi-key attacks
• vulnerable to padding oracle attacks
• not compatible with nip4. if an upgrade is required, it should be a proper upgrade

i had various problems using chacha

Which problems, exactly?

Dropping an idea I had around metadata concerns that would not alter this NIP as proposed, because if this is more cryptographically sound than NIP04, it should move forward on that basis alone.

Metadata can be addressed in a separate NIP, using this one as a dependency to keep those concerns separated.

"Chaum Chat"

The objection to ephemeral conversation keys ties back to paid relays not allowing writes for new keys. With all he stupidity being pushed by NGO's to use ECash in representing custodial Bitcoin, why not re-purpose those libraries into something useful, like relay access tokens?

An additive NIP for paid relays could define how clients receive ECash instead of a whitelisting, with the ECash then being used for the whitelisting of keys.

Additionally, free relays could reward ECash to identity keys based on reputation, to keep rate limits loose without abuse.

This process would decouple identity keys and their ephemeral children from the Lightning payment.

Since's it's only for service access, there's no dishonest NGO narrative of it enhancing the security/privacy of Bitcoin transactions themselves. It also serves as a demonstration of how ECash should be used.

"Chaum Chat"

I can't evaluate the merits of this idea without a full description. But I am happy to do so if you decide to make a NIP for it. It feels like Large Group chats (~1,000+ people) could use this Ecash allowlist idea.

Correct me if I am wrong, but I have a question regarding this NIP as well as NIP-04. So the ECDH part involves computation using the private key. Where exactly should this computation getConversationKey take place?

Ideally, users should be able to store their private key in an extension wallet such as Metamask of Ethereum. But Metamask won't let a Dapp export your private key for an ECDH computation. For Nostr, idealy it should be same—web app can't touch your private key directly. Maybe we should have a Nostr wallet which does signature and ECDH computation within the wallet instead of exporting it to a web app. Thoughts?

Of course. That was one of ideas behind getConversationKey. It can be abstracted in a proper way.

I've created a new version of this PR at #715, @paulmillr please let me know if you'd like to close this PR or incorporate those changes into this one.

Closing this since the conversation is continuing at #715